Generic Finite Element Programming for Massively Parallel Flow Simulations

We present design issues, data structures and algorithms which allow us to leverage the power of high performance computer clusters in generic finite element codes. Large scale flow simulations require parallel algorithms to scale to thousands of processor cores. Simple parallelization in generic finite element codes often reduces to implementing parallel linear algebra and solvers or interfacing with an existing parallel linear algebra library. Other data structures and algorithms, e.g. mesh handling, are not designed for parallel computations and introduce bad scaling and memory constraints. Scaling to more than about one hundred cores and a few million of unknowns is not feasible. Today’s computer clusters have up to tens of thousands of cores and are the foundation to deal with large scale numerical problems. Generic finite element codes, like deal.II , feature methods for rapid and flexible development, higher order elements and adaptivity. To take advantage of those features in large parallel computer clusters, e.g. in flow simulations, many parts of the library have to be adapted and tailored to scale. We describe the steps done in enabling deal.II to scale from less than a hundred cores and a few million unknowns up to thousands of cores and billions of unknowns: data structures for distributed fully-adaptive mesh handling, data structures for efficient indexing of degrees of freedom, algorithms for efficiently distributing the degrees of freedom, and more. Distributing storage of data and local – instead of global – communication plays a key role in this process. The parallelization is done in a general setting applicable to any generic finite element library, c.f. . We show numerical results that demonstrate the applicability of our design to generic finite element problems, and that our algorithms scale very efficiently to billions of unknowns.